Knowledge Base Population using Semantic Label Propagation

A crucial aspect of a knowledge base population system that extracts new facts from text corpora, is the generation of training data for its relation extractors. In this paper, we present a method that maximizes the effectiveness of newly trained relation extractors at a minimal annotation cost. Manual labeling can be significantly reduced by Distant Supervision (DS), which is a method to construct training data automatically by aligning a large text corpus with an existing knowledge base of known facts. For example, all sentences mentioning both ‘Barack Obama’ and ‘US’ may serve as positive training instances for the relation born in(subject,object). However, distant supervision typically results in a highly noisy training set: many training sentences containing the known entity pairs do not really express the intended relation. We explore the idea of combining DS with (partial) human supervision to eliminate that noise. This idea is not novel per se, but our key contributions are: (i) a novel method of filtering the DS training set based on labeling Shortest Dependency Paths, (SDPs), and (ii) the Semantic Label Propagation (SLP) model. We propose to combine DS with minimal manual human supervision by annotating features (in particular SDPs) rather than (potential) relation instances. Such so-called feature labeling is adopted to eliminate noise from the large and noisy initial training set, resulting in a significant increase of precision (at the expense of recall). We further improve on this approach by introducing the Semantic Label Propagation (SLP) method, which uses the similarity between low-dimensional representations of candidate training instances, to extend the (filtered) training set in order to increase recall while maintaining high precision. Our proposed strategy for generating training data is studied and evaluated on an established test collection designed for knowledge base population (KBP) tasks from the TAC KBP English slot filling task. The experimental results show that the SLP strategy leads to substantial performance gains when compared to ∗Corresponding author Email address: [email protected] (Lucas Sterckx) Preprint submitted to Knowledge Based Systems March 4, 2016 ar X iv :1 51 1. 06 21 9v 2 [ cs .C L ] 3 M ar 2 01 6 existing approaches, while requiring an almost negligible human annotation effort.